Papers with Cross-lingual transfer learning
An Empirical Study on Cross-X Transfer for Legal Judgment Prediction (2022.aacl-main)
Copied to clipboard
| Challenge: | Cross-lingual transfer learning is understudied in legal NLP but not in legal Judgment Prediction (LJP). |
| Approach: | They explore cross-lingual transfer learning techniques on legal JP using a trilingual Swiss-Judgment-Prediction dataset and adapter-based fine-tuning. |
| Outcome: | The proposed methods improve the model’s performance by augmenting the training dataset with machine-translated versions of the original documents, using a 3 larger training corpus. |
Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddings (2023.emnlp-main)
Copied to clipboard
| Challenge: | XLMs can support cross-lingual transfer learning with little to no additional training data. |
| Approach: | They describe a mechanism for cross-lingual transfer learning by measuring the properties of the initial token embedding layer. |
| Outcome: | The proposed model can be used to support cross-lingual transfer learning . the initial token embedding layer is expressive and interpretable . |
Domain Mismatch Doesn’t Always Prevent Cross-lingual Transfer Learning (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies have reported that domain mismatch prevents cross-lingual transfer . UBLI and UNMT do not work well when underlying monolingual corpora come from different domains . |
| Approach: | They show that a simple initialization regimen can overcome domain mismatch in cross-lingual transfer . they pre-train word embeddings on concatenated domain-mismatched corpora and use them as initializations . |
| Outcome: | The initialization regimen can overcome the domain mismatch effect in cross-lingual transfer learning . the initializations were used for MUSE UBLI, UN Parallel UNMT, and the SemEval 2017 task . |
Struct-XLM: A Structure Discovery Multilingual Language Model for Enhancing Cross-lingual Transfer through Reinforcement Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods require syntactic labels that are difficult to obtain and of poor quality for low-resource languages. |
| Approach: | They propose a syntactic alignment model that leverages reinforcement learning to discover universal syntaktic structures for cross-lingual PLM alignment. |
| Outcome: | The proposed model improves cross-lingual representation alignment on the XTREME benchmark. |
Vicinal Risk Minimization for Few-Shot Cross-lingual Transfer in Abusive Language Detection (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for few-shot cross-lingual transfer learning are limited in target languages due to the scarcity of resources. |
| Approach: | They propose a method which interpolates pairs of instances based on the angle of their representations and propose augmentation methods to enhance few-shot cross-lingual abusive language detection. |
| Outcome: | The proposed method improves few-shot cross-lingual abusive language detection in seven languages typologically distinct from English and three different domains. |
Multi-Source Cross-Lingual Model Transfer: Learning What to Share (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual transfer learning (CLTL) is a viable method for building NLP models for a low-resource target language . however, many languages lack the labeled training data necessary for training deep neural nets for varying NLP tasks. |
| Approach: | They propose a cross-lingual transfer learning method that leverages annotated data from other languages to build NLP models for a target language. |
| Outcome: | The proposed model achieves significant performance gains over prior art over multiple text classification and sequence tagging tasks including a large-scale industry dataset. |
Cross-lingual Transfer Learning for Grammatical Error Correction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies on English GEC have focused on improving it, but the resources required to train the models are not sufficient. |
| Approach: | They investigate cross-lingual transfer learning in grammatical error correction tasks . similarities between these languages is a key factor for successfully transferring grammatikal knowledge . |
| Outcome: | The proposed methods improve accuracy of grammatical error correction tasks in English and Russian, but lack the resources to train models in these languages. |
Make the Best of Cross-lingual Transfer: Evidence from POS Tagging with over 100 Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual generalisability of large pre-trained models use English training data and test data in unseen languages. |
| Approach: | They propose to use multilingual pre-trained models to model cross-lingual transfer in a selection of target languages. |
| Outcome: | The proposed model can be used to improve cross-lingual transfer performance in low-resource languages with no labeled training data. |
Transitioning Representations between Languages for Cross-lingual Event Detection via Langevin Dynamics (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets for event detection (ED) are limited to a small set of popular languages due to the high cost of data annotation. |
| Approach: | They propose a method to develop cross-lingual transfer learning models in high-resource source languages . they aim to transition the representations for target-language examples into the source-language space . |
| Outcome: | Extensive experiments over three languages demonstrate the state-of-the-art in CLTL. |